Method and Apparatus for Mobile Stroke Self-Detection

ABSTRACT

Disclosed herein are implementations of a method and apparatus for stroke self-detection. The method and apparatus may include a mobile platform for stroke detection. The method may include receiving sensor data. The method may include comparing the sensor data with a baseline test result to determine a test score. The method may include determining a passing test result based on a threshold. The method may include transmitting the results or an alert to one or more of an emergency contact, emergency medical services, a physician, or a telemedicine provider.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation-in-part of U.S. patent applicationSer. No. 16/861,363, filed Apr. 29, 2020, which claims priority to andthe benefit of U.S. Provisional Patent Application No. 62/842,952, filedMay 3, 2019, the entire disclosure of which is hereby incorporated byreference.

SUMMARY

Disclosed herein are implementations of a method and apparatus forstroke self-detection. The method and apparatus may include a mobileplatform for stroke self-detection. The apparatus may be configured toreceive sensor data. The apparatus may be configured to compare thesensor data with a baseline test result to determine a test score. Theapparatus may be configured to determine a passing test result based ona threshold.

In an aspect, a computing device for stroke detection may include adisplay, a plurality of sensors, a memory, and a processor. The displaymay be configured to display an instruction. The plurality of sensorsmay include an accelerometer, a camera, and a microphone. The processormay be configured to obtain sensor data based on the instruction. Thesensor data may include accelerometer data from the accelerometer, imagecapture data from the camera, and microphone data from the microphone.The microphone data may include voice data associated with a patientresponse. The processor may be configured to convert the voice data totext data. The processor may be configured to determine a strokeself-detection score based on the obtained sensor data. Thedetermination of the stroke self-detection score may include acomparison of the text data to data associated with the instruction anda determination of a cognitive score. The instruction may include arequest for patient age. The determination of the cognitive score mayinclude a determination that the cognitive score is zero (0) on acondition that the text data matches the data associated with theinstruction. The determination of the cognitive score may include adetermination that the cognitive score is one (1) on a condition thatthe text data does not match the data associated with the instruction.The computing device may be configured to store the strokeself-detection score in the memory. The computing device may beconfigured to display the stroke self-detection score on the display asa result summary. The computing device may be configured to transmit analert when the stroke self-detection score is above a threshold.

In an aspect, a computing device may include a display, a microphone, amemory, and a processor. The display may be configured to display aninstruction. The processor may be configured to obtain microphone datafrom the microphone. The microphone data may include voice dataassociated with a patient response. The processor may be configured toconvert the voice data to text data. The processor may be configured todetermine a stroke self-detection score based on a comparison of thetext data to data associated with the instruction and a determination ofa cognitive score. The instruction may include a request for a currentmonth. The determination of the cognitive score may include adetermination that the cognitive score is zero (0) on a condition thatthe text data matches the data associated with the instruction. Thedetermination of the cognitive score may include a determination thatthe cognitive score is one (1) on a condition that the text data doesnot match the data associated with the instruction. The computing devicemay be configured to store the stroke self-detection score in thememory. The computing device may be configured to display the strokeself-detection score on the display as a result summary. The computingdevice may be configured to transmit an alert if the strokeself-detection score is above a threshold.

In an aspect, a method for stroke self-detection may include displayingan instruction on a display. The method may include obtaining sensordata. The sensor data may be based on the instruction. The sensor datamay include accelerometer data, image capture data, microphone data, orany combination thereof. The method may include determining a strokeself-detection score. The stroke self-detection score may be based onthe obtained sensor data. The method may include storing the strokeself-detection score in a memory. The method may include displaying thestroke self-detection score on the display as a result summary. Themethod may include transmitting an alert if the stroke self-detectionscore is above a threshold.

In an aspect, a method for stroke self-detection may include displayingan instruction on a display. The method may include obtaining sensordata based on the instruction. The sensor data may include microphonedata. The microphone data may include voice data associated with apatient response. The method may include converting the voice data totext data. The method may include determining a stroke self-detectionscore by comparing the text data to data associated with theinstruction. The method may include storing the stroke self-detectionscore in a memory. The method may include displaying the strokeself-detection score on the display as a result summary. The method mayinclude transmitting an alert if the stroke self-detection score isabove a threshold.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosure is best understood from the following detaileddescription when read in conjunction with the accompanying drawings. Itis emphasized that, according to common practice, the various featuresof the drawings are not to-scale. On the contrary, the dimensions of thevarious features are arbitrarily expanded or reduced for clarity.

FIG. 1 is a diagram of an example of a computing device in accordancewith implementations of this disclosure.

FIG. 2 is a diagram of an example of the processor shown in FIG. 1 inaccordance with implementations of this disclosure.

FIG. 3 is a flow diagram of an example of a method for strokeself-detection in accordance with embodiments of this disclosure.

FIG. 4 is a flow diagram of another example of a method for strokeself-detection in accordance with embodiments of this disclosure.

FIG. 5A is a diagram of an example of an image for face detection.

FIG. 5B is a diagram of the image shown in FIG. 5A enlarged to showpixel distance based on pupil position.

FIG. 6 is a flow diagram of another example of a method 600 for strokeself-detection in accordance with embodiments of this disclosure.

FIG. 7 is a diagram of an example display generated by the numberdetection module shown in FIG. 2 .

FIG. 8 is a flow diagram of another example of a method for strokeself-detection in accordance with embodiments of this disclosure.

FIG. 9 is a diagram of an example of an image used for smile detection.

FIG. 10 is a flow diagram of another example of a method for strokeself-detection in accordance with embodiments of this disclosure.

FIG. 11 is a flow diagram of another example of a method for strokeself-detection in accordance with embodiments of this disclosure.

FIG. 12 is a flow diagram of another example of a method for strokeself-detection in accordance with embodiments of this disclosure.

FIG. 13 is a diagram of an example display generated by the objectdetection module shown in FIG. 2 .

FIG. 14 is a flow diagram of another example of a method for strokeself-detection in accordance with embodiments of this disclosure.

FIG. 15 is a flow diagram of an example scoring method that may be usedby any of the embodiments disclosed herein.

DETAILED DESCRIPTION

Over 800,000 strokes occur yearly, and are the leading cause ofdisability in the United States. Patients lose two million neurons persecond during a stroke. 20-25% of patients get help within 3 hours.Typical stroke detection methods require a physician, and the majorityof patients do not make it to the hospital soon enough to receiveadequate treatment. Accordingly, it would be desirable to have a methodand apparatus for a patient to self-detect a stroke without the need ofa physician being present.

The systems and methods described herein may be used by a patient toself-detect a stroke without the need of a physician being present. Thestroke self-detection systems and methods may be based on the NationalInstitute of Health stroke scale.

As used herein, the terminology “computer” or “computing device”includes any unit, or combination of units, capable of performing anymethod, or any portion or portions thereof, disclosed herein.

As used herein, the terminology “processor” indicates one or moreprocessors, such as one or more special purpose processors, one or moredigital signal processors, one or more microprocessors, one or morecontrollers, one or more microcontrollers, one or more applicationprocessors, one or more central processing units (CPU)s, one or moregraphics processing units (GPU)s, one or more digital signal processors(DSP)s, one or more application specific integrated circuits (ASIC)s,one or more application specific standard products, one or more fieldprogrammable gate arrays, any other type or combination of integratedcircuits, one or more state machines, cloud-based computing processors,or any combination thereof.

As used herein, the terminology “memory” indicates any non-transitorycomputer-usable or computer-readable medium or device that can tangiblycontain, store, communicate, or transport any signal or information thatmay be used by or in connection with any processor. For example, amemory may be one or more read only memories (ROM), one or more randomaccess memories (RAM), one or more registers, low power double data rate(LPDDR) memories, one or more cache memories, one or more semiconductormemory devices, one or more magnetic media, one or more optical media,one or more magneto-optical media, or any combination thereof.

As used herein, the terminology “instructions” may include directions orexpressions for performing any method, or any portion or portionsthereof, disclosed herein, and may be realized in hardware, software,cloud-based computing environment(s), or any combination thereof. Forexample, instructions may be implemented as information, such as acomputer program, stored in memory that may be executed by a processorto perform any of the respective methods, algorithms, aspects, orcombinations thereof, as described herein. Instructions, or a portionthereof, may be implemented as a special purpose processor, orcircuitry, that may include specialized hardware for carrying out any ofthe methods, algorithms, aspects, or combinations thereof, as describedherein. In some implementations, portions of the instructions may bedistributed across multiple processors on a single device, on multipledevices, which may communicate directly or across a network such as alocal area network, a wide area network, the Internet, or a combinationthereof.

As used herein, the terminology “determine” and “identify,” or anyvariations thereof, includes selecting, ascertaining, computing, lookingup, receiving, determining, establishing, obtaining, or otherwiseidentifying or determining in any manner whatsoever using one or more ofthe devices and methods shown and described herein.

As used herein, the terminology “example,” “embodiment,”“implementation,” “aspect,” “feature,” or “element” indicates serving asan example, instance, or illustration. Unless expressly indicated, anyexample, embodiment, implementation, aspect, feature, or element isindependent of each other example, embodiment, implementation, aspect,feature, or element and may be used in combination with any otherexample, embodiment, implementation, aspect, feature, or element.

As used herein, the terminology “or” is intended to mean an inclusive“or” rather than an exclusive “or.” That is, unless specified otherwise,or clear from context, “X includes A or B” is intended to indicate anyof the natural inclusive permutations. That is, if X includes A; Xincludes B; or X includes both A and B, then “X includes A or B” issatisfied under any of the foregoing instances. In addition, thearticles “a” and “an” as used in this application and the appendedclaims should generally be construed to mean “one or more” unlessspecified otherwise or clear from the context to be directed to asingular form.

Further, for simplicity of explanation, although the figures anddescriptions herein may include sequences or series of steps or stages,elements of the methods disclosed herein may occur in various orders orconcurrently. Additionally, elements of the methods disclosed herein mayoccur with other elements not explicitly presented and described herein.Furthermore, not all elements of the methods described herein may berequired to implement a method in accordance with this disclosure.Although aspects, features, and elements are described herein inparticular combinations, each aspect, feature, or element may be usedindependently or in various combinations with or without other aspects,features, and elements.

One or more embodiments disclosed herein may include an example of aplatform configured to allow a patient to self-detect a stroke. Theplatform may be implemented on any computing device, for example, amobile device such as a mobile telephone, a tablet, or a wearabledevice. The platform includes a user interface. The user interface mayinclude a touch display. The platform may obtain information via theuser interface. The information may include patient information,including, for example, patient name, address, age, height, weight, andgender. The information may include an emergency contact name, contactnumber, physician name, physician number, or any combination thereof.The information may include past medical history, medication, or both.

The platform may be configured to perform one or more tests via the userinterface. One or more tests may use facial recognition, speechanalysis, object recognition, motion recognition, or any combinationthereof. The one or more tests may use one or more sensors of the mobiledevice, including, for example, a camera, depth sensor, accelerometer,gyroscope, a global positioning system (GPS), microphone, or anycombination thereof. For example, the camera, depth sensor, or both, maybe used to obtain a point cloud from one or more images to determine aface for facial recognition. In another example, the camera may be usedeither alone or in conjunction with the accelerometer to determinemotion. The one or more tests may include a baseline test. The baselinetest may be performed any number of times to determine a baseline foreach patient. Any subsequent test result may be compared with thebaseline test result to determine whether the patient is experiencing astroke. In some implementations, the platform may be configured toautomatically contact the patient's physician if it is determined thatthe patient is experiencing a stroke. Patient data may be stored on thedevice, on a cloud server, or both. The patient data may include patientprofile data and test data. The patient profile data may include thepatient's name, medical history, medications, allergies, emergencycontact information, physician contact information, or any combinationthereof.

An example of voice recognition may include obtaining voice samples fromthe patient of known words or phrases and comparing the obtained voicesamples with the patient's baseline test result. The system maydetermine a score based on the baseline test result. For example, ascore close to 1 may be normal, and a score close to 0 may be indicativeof a complete mismatch, i.e. a high risk for a stroke.

The platform may be configured to generate displays of test results. Forexample, a pass indication and a fail indication for each test conductedmay be displayed on the user interface. Any indication may be used toshow pass or fail. For example, the pass indication may be shown as acheck mark, and the fail indication may be shown as an “X.” In someexamples, the test results may trigger the summary display. If thenumber of abnormal test results is above a threshold, the platform mayautomatically contact the patient's physician, emergency services, oneor more of the patient's emergency contacts, or any combination thereof.The threshold may be any value that is above a patient's baseline.

FIG. 1 is a diagram of an example of a computing device 100 that may beused to implement any of the systems and platforms described herein. Thecomputing device 100 may be a hardware device, or it may be implementedin software as a virtual machine. The computing device 100 may include asmartphone, tablet, wearable device, or personal computing device. Thecomputing device 100 includes a memory 105, a processor 110, a display115, one or more motors 120, a transceiver 125, an accelerometer 130,one or more speakers 135, one or more image sensors 140, one or moremicrophones 145, one or more antennas 150, or any combination thereof.For example, some implementations of the computing device 100 may notinclude the display 115.

The memory 105 may include a system memory module that is configured tostore executable computer instructions that, when executed by processor,control various functions of the computing device. The memory mayinclude non-transitory memory configured to store patient data,self-detection request data, self-detection response data, or anycombination thereof. Patient data may include patient medical historydata, emergency contact information, physician contact information,patient address data, patient medical insurance data, or any combinationthereof. Self-detection request data may include request data to solicitpatient input for self-detection purposes. The self-detection requestdata may include text data, text-to-speech data, or both. Self-detectionresponse data may include patient voice data obtained in response to theself-detection request. The self-detection response data may includespeech-to-text data, voice data, or both.

The processor 110 may include a system on a chip (SOC) microcontroller,microprocessor, CPU, DSP, ASIC, GPU, or other processors that controlthe operation and functionality of the computing device. The processor110 may interface with mechanical, electrical, sensory, and powermodules via driver interfaces and software abstraction layers.Additional processing and memory capacity may be used to support theseprocesses. These components may be fully controlled by the processor110. In some implementations, one or more components may be operable byone or more other control processes in accordance with a given schedule.The memory 105 may include a database that is configured to storeinformation from the processor 110. The processor 110 may be configuredto receive an electrical signal associated with an audible sound (e.g.,a voice input) from the one or more microphones 145, and convert theaudible sound to text. The processor 110 may be configured to obtain atext instruction, convert the text instruction to a speech signal, andtransmit the speech signal to the one or more speakers 135.

The display 115 may include an interactive touch display. The display115 may be a liquid crystal display (LCD) display unit, a light-emittingdiode (LED) display unit, an organic light-emitting diode (OLED) displayunit, or a micro organic light-emitting diode (micro-OLED) display unit.The display 115 may be a capacitive display that is configured toreceive a user input, for example, via a touch or gesture.

The one or more motors 120 may include an eccentric rotating motor(ERM), a linear resonant actuator (LRA), or both. The one or more motors120 are configured to vibrate to provide haptic feedback. The one ormore motors 120 may collectively be referred to as a haptic engine or ataptic engine. The one or more motors 120 may receive a signal from theprocessor 110 causing the one or more motors 120 to vibrate. The signalreceived from the processor 110 may be a control signal. The controlsignal may be received via one or more haptic drivers. In an example,the control signal from the processor 110 may cause the one or morehaptic drivers to transmit a current to the one or more motors 120. Thecurrent may be modulated to vary the strength of the vibrationsgenerated by the one or more motors 120.

The transceiver 125 is coupled to the processor 110 and the one or moreantennas 150. Although the transceiver 125 is shown as a single unit,some embodiments may implement the transceiver 125 as a separatereceiver unit and transmitter unit. While FIG. 1 depicts the processor110 and the transceiver 125 as separate components, it will beappreciated that the processor 110 and the transceiver 125 may beintegrated together in an electronic package or chip. The transceiver125 may be configured to modulate signals that are to be transmitted bythe one or more antennas 150 and to demodulate the signals that arereceived by the one or more antennas 150. The transceiver 125 mayinclude multiple transceivers for enabling the computing device 100 tocommunicate via multiple radio access technologies.

The accelerometer 130 may be a single-axis or multi-axis component thatis configured to detect magnitude and direction of the properacceleration as a vector quantity. The magnitude and direction of theproper acceleration may be used to sense orientation, coordinateacceleration, vibration, shock, and falling in a resistive medium. Forexample, the accelerometer 130 may be configured to detect whether anextended arm of a patient is shaking or drifting in a downward orsideways direction and generate and transmit a signal to the processor110. The processor 110 may be configured to determine whether anextended arm of a patient is shaking or drifting in a downward orsideways direction based on the accelerometer signal. In someimplementations, the accelerometer 130 may be a micromachinedmicroelectromechanical system (MEMS) accelerometer configured to detectthe position of the computing device 100 and provide input for strokeself-detection and determination.

The one or more speakers 135 may each be an electroacoustic transducerconfigured to convert an electrical audio signal from the processor 110into a corresponding sound in the audible frequency range (e.g., about20 Hz to about 20 KHz). In one or more embodiments, the one or morespeakers 135 may be configured to transmit sound in the form of a voicerequest to illicit a user response.

The one or more image sensors 140 are configured to detect and conveyinformation used to make an image. The one or more image sensors 140 maybe configured to convert the variable attenuation of light waves intosignals that convey the information. The waves may be light or otherelectromagnetic radiation. The one or more image sensors 140 may includedigital cameras, depth sensors, infrared (IR) sensors, or anycombination thereof. The one or more image sensors 140 may be configuredto capture images, video, or both.

The one or more microphones 145 may each be a transducer configured toconvert an audible sound into an electrical signal. The one or moremicrophones 145 may include a dynamic microphone, a condensermicrophone, a piezoelectric microphone, or any combination thereof.

The one or more antennas 150 may be configured to transmit signals to,or receive signals from, a wireless device, such as a base station, overan air interface. For example, in one embodiment, the one or moreantennas 150 may be configured to transmit and/or receive radiofrequency (RF) signals. In another embodiment, the one or more antennas150 may be an emitter/detector configured to transmit and/or receive IR,ultraviolet (UV), or visible light signals, for example. In yet anotherembodiment, the one or more antennas 150 may be configured to transmitand receive both RF and light signals. It will be appreciated that theone or more antennas 150 may be configured to transmit and/or receiveany combination of wireless signals.

FIG. 2 is a block diagram of an example of the processor 110 shown inFIG. 1 . As shown in FIG. 2 , the processor 110 includes a cognitivedetection module 210, an eye movement detection module 220, a numberdetection module 230, a smile detection module 240, an arm motiondetection module 250, a leg motion detection module 260, a vibrationdetection module 270, an object detection module 280, and a sentencedetection module 290. The processor 110 may be configured to executeinstructions from a non-transitory computer readable medium based on thecognitive detection module 210, the eye movement detection module 220,the number detection module 230, the smile detection module 240, the armmotion detection module 250, the leg motion detection module 260, thevibration detection module 270, the object detection module 280, thesentence detection module 290, or any combination thereof. Each moduleshown in FIG. 2 is configured to determine one or more respective strokeself-detection scores. One or more of the determined strokeself-detection scores may be summed to determine an overall score.

FIG. 3 is a flow diagram of an example of a method 300 for strokeself-detection in accordance with embodiments of this disclosure. Themethod 300 may be performed by the processor 110 shown in FIGS. 1 and 2. In this example, the method 300 may be performed by the cognitivedetection module 210 shown in FIG. 2 .

As shown in FIG. 3 , the method 300 includes displaying and speaking 310a request for patient age. For example, the cognitive detection module210 may cause the processor 110 to send a signal to the display 115shown in FIG. 1 to display text of the request for patient age, send asignal to the one or more speakers 135 shown in FIG. 1 to speak therequest for patient age, or both. In an example, the text “How old areyou?” may be displayed on the display 115 and the audible phrase “Howold are you?” may be emitted from the one or more speakers 135.

The method 300 includes receiving 320 a voice response. The voiceresponse may be received by the one or more microphones 145 shown inFIG. 1 . The voice response may be processed using any open sourcespeech recognition technique. The voice response may be received as anaudible phrase such as, for example, “forty-five,” “forty-five years,”or “I am forty-five years old.” The one or more microphones 145 may beconfigured to transmit a signal associated with the voice response tothe processor 110. The cognitive determination module 210 may cause theprocessor to convert the received signal associated with the voiceresponse to text and compare the text to age data in a user profile.

The method 300 includes determining 330 a score, for example, a strokeself-detection score. The score may be referred to as a cognitive scoreor a cognition detection score. The score may be based on adetermination of whether the text associated with the voice responsematches the age data in the user profile. If the text associated withthe voice response matches the age data in the user profile, adetermination is made that the response was correct, and a score of zero(0) is determined. If the text associated with the voice response doesnot match the age data in the user profile, a determination is made thatthe response was incorrect, and a score of one (1) is determined. Thescore may be stored, for example in memory 105 shown in FIG. 1 , forlater calculation and tabulation.

The method 300 includes displaying and speaking 340 a request for thecurrent month. For example, the cognitive detection module 210 may causethe processor 110 to send a signal to the display 115 shown in FIG. 1 todisplay text of the request for the current month, send a signal to theone or more speakers 135 shown in FIG. 1 to speak the request for thecurrent month, or both. In an example, the text “What month is it?” maybe displayed on the display 115 and the audible phrase “What month isit?” may be emitted from the one or more speakers 135.

The method 300 includes receiving 350 a voice response. The voiceresponse may be received by the one or more microphones 145 shown inFIG. 1 . The voice response may be received as an audible phrase suchas, for example, “April” or “It is April.” The one or more microphones145 may be configured to transmit a signal associated with the voiceresponse to the processor 110. The cognitive determination module 210may cause the processor to convert the received signal associated withthe voice response to text and compare the text to the current date, forexample, in a calendar module.

The method 300 includes determining 360 a score, for example, a strokeself-detection score. The score may be referred to as a cognitive scoreor a cognition detection score. The score may be based on adetermination of whether the text associated with the voice responsematches the current date. If the text associated with the voice responsematches the current date, a determination is made that the response wascorrect, and a score of zero (0) is determined. If the text associatedwith the voice response does not match the current date, a determinationis made that the response was incorrect, and a score of one (1) isdetermined. The score may be stored, for example in memory 105 shown inFIG. 1 , for later calculation and tabulation.

FIG. 4 is a flow diagram of another example of a method 400 for strokeself-detection in accordance with embodiments of this disclosure. Themethod 400 may be performed by the processor 110 shown in FIGS. 1 and 2. In this example, the method 400 may be performed by the eye movementdetection module 220 shown in FIG. 2 .

As shown in FIG. 4 , the method 400 includes displaying and speaking 410an instruction. For example, the eye movement detection module 220 maycause the processor 110 to send a signal to the display 115 shown inFIG. 1 to display text of the instruction, send a signal to the one ormore speakers 135 shown in FIG. 1 to speak the instruction, or both. Inan example, the text “Keep your head still. With your eyes only, look tothe left and look to the right.” may be displayed on the display 115 andthe audible phrase “Keep your head still. With your eyes only, look tothe left and look to the right.” may be emitted from the one or morespeakers 135.

The eye movement detection module 220 may cause the processor 110 tosend a signal to the one or more image sensors 140 shown in FIG. 1 toinitiate a video recording of the face of the patient. In someimplementations, the video recording may be stored in the memory 105shown in FIG. 1 . The processor 110 may obtain images 420 from the videorecording. The processor 110 may perform face and face landmarkdetection, text detection, image registration, and general featuretracking using any open source technique. The processor 110 may usemachine learning (ML) models for tasks such as classification or objectdetection using any open source technique. The processor 110 may beconfigured to track multiple objects, such as a patient's pupils, orrectangles throughout the video recording.

The processor 110 may identify and segment objects of the patient'sface, for example, eyes, eyebrows, mouth, nose, or any combinationthereof. Once the eyes are detected and segmented, the processor 110 maydetect 430 pupil position. In an example, the iris and the pupil may bedetected as one object. The processor 110 may detect the pupil positionin one or more frames of the video recording. For each frame that thepupil position is detected, the processor 110 calculates 440 a pixeldistance from the outer edge of the pupil to the corner of the eye inthe direction of the eye movement (i.e., the left corner of the left eyeor the right corner of the right eye). An indication that the patient isable to move their eyes to each side may be that the respective pupil isclose to the edge of the respective eye.

The eye movement detection module 220 may cause the processor 110 todetermine 450 a score, for example, a stroke self-detection score, basedon the calculated pixel distance. The score may be referred to as an eyemovement score or an eye movement detection score. The score may bebased on the frame that has the smallest pixel distance. In an example,the processor 110 may determine a score of zero (0) if the pixeldistance is less than 2 pixels. If the pixel distance is greater than 2pixels when the patient looks to either the left side or the right side,the processor may determine a score of one (1). If the pixel distance isgreater than 2 pixels when the patient looks to the left side and theright side, the processor 110 may determine a score of two (2). Forceddeviation of eyes to one side may also result in the processor 110determining a score of two (2). In an example of forced deviation, ifthe pixel distance is less than 2 pixels when the patient looks toeither the left side or the right side and, simultaneously, the pixeldistance of the opposite side is greater than 15 pixels, the processor110 may determine a score of two (2). The score may be stored, forexample in memory 105 shown in FIG. 1 , for later calculation andtabulation.

In another example, the total score for eye movements may be determinedby individual scores for each eye. The processor 110 may record thesmallest distance between the left pupil and the leftmost corner of theleft eye. Similarly, the processor 110 may record the smallest distancebetween the right pupil and the rightmost corner of the right eye. Inthis example, an example passing score may be 0.07 for the left eye, and0.1 for the right eye. The scores are based on the relative positioningof the pupil inside a 1.0×1.0 bounding box that an eye tracking computervision algorithm places around the eye. For the eyes, there are no unitsof measurement. Software running on the processor 110 can be used todetect the patient's facial feature (i.e., eye) and places a boundingbox around the feature. The software normalizes the coordinates to thedimensions of the bounding box. For the eyes, the processor 110 recordsthe x data points, which track where the pupil is located in relation tothe bounding box surrounding the eye. The origin (0.0, 0.0) is at thelower-left corner of the bounding box, and the upper-right corner of thebounding box is (1.0, 1.0), similar to an (x, y) coordinate system.

If either the smallest left eye distance is greater than the passingscore or the smallest right eye distance is greater than the passingscore, then the patient receives a total score of one (1). If both thesmallest left eye distance is greater than the passing score and thesmallest right eye distance is greater than the passing score, then thepatient receives a total score of two (2). If one eye receives a passingscore and the other eye receives a failing score, then the patientreceives a total score of two (2). If the patient receives a passingscore on both eyes, then the patient receives a total score of zero (0).

FIG. 5A is a diagram of an example of an image 500 for face detection.Detecting objects of the patient's face may include detecting alldetectable two-dimensional (2D) face landmarks and regions, and exposingthe face landmarks and regions as properties. As shown in FIG. 5A, thecoordinates of the face landmarks may be normalized to the dimensions ofa face bounding box 510, with the origin at the bounding box'slower-left corner. An image point function may be used to convertnormalized face landmark points into absolute points within thecoordinate system of the image or frame. As shown in FIG. 5A, detectedobjects (shown in bounding boxes as dashed lines) of the patient's facemay include, for example, eyes 520, eyebrows 530, and nose 540.

FIG. 5B is a diagram of the image 500 enlarged to show pixel distancebased on pupil position. In this example, the patient's right eye isshown. The direction of eye movement is shown with arrow 550 when thepatient looks to the right. As shown in FIG. 5A, the eye 520 includes aninner corner 560, and outer corner 570, and a pupil portion 580, whichmay include the iris. In this example, the pupil portion 580 issegmented and includes the iris portion shown in a dashed line. As theright eye moves to the right, the edge of the pupil portion 580approaches the outer corner 570 of the right eye. The pixel distance 590is shown to be the distance in pixels between the outer corner 570 ofthe eye 520 and the edge of the pupil portion 580.

FIG. 6 is a flow diagram of another example of a method 600 for strokeself-detection in accordance with embodiments of this disclosure. Themethod 600 may be performed by the processor 110 shown in FIGS. 1 and 2. In this example, the method 600 may be performed by the numberdetection module 230 shown in FIG. 2 .

As shown in FIG. 6 , the method 600 includes displaying and speaking 610an instruction. For example, the number detection module 230 may causethe processor 110 to send a signal to the display 115 shown in FIG. 1 todisplay text of the instruction, send a signal to the one or morespeakers 135 shown in FIG. 1 to speak the instruction, or both. In anexample for the right hand, the text “Place the phone in your right handand hold the phone away from your at arm's length. Have the screenfacing you.” may be displayed on the display 115 and the audible phrase“Place the phone in your right hand and hold the phone away from your atarm's length. Have the screen facing you.” may be emitted from the oneor more speakers 135. In this example, the text “Keep your head stilland look straight ahead. Do not move your eyes.” may be displayed on thedisplay 115 and the audible phrase “Keep your head still and lookstraight ahead. Do not move your eyes.” may be emitted from the one ormore speakers 135.

The number detection module 230 may cause the processor 110 to send asignal to the display 115 shown in FIG. 1 to display 620 a number. In anexample, the number may be displayed as an image or using text. The text“Without moving your eyes, what number do you see?” may be displayed onthe display 115 and the audible phrase “Without moving your eyes, whatnumber do you see?” may be emitted from the one or more speakers 135. Insome embodiments, the processor 110 may determine 630 whether eyemovement is detected. The detection of eye movement may be performed asdescribed in FIG. 4 above.

The method 600 includes receiving 640 a voice response. The voiceresponse may be received by the one or more microphones 145 shown inFIG. 1 . The voice response may be processed using any open sourcespeech recognition technique. The voice response may be received as anaudible phrase such as, for example, “two,” “three,” or “four.” The oneor more microphones 145 may be configured to transmit a signalassociated with the voice response to the processor 110. The numberdetection module 230 may cause the processor 110 to convert the receivedsignal associated with the voice response to text and compare the textto data associated with the displayed number.

The method 600 includes determining 650 a score, for example, a strokeself-detection score. The score may be referred to as a number detectionscore or a visual perception score. The score may be based on adetermination of whether the text associated with the voice responsematches the data associated with the displayed number. If the textassociated with the voice response matches the data associated with thedisplayed number, a determination is made that the response was correct,and a score of zero (0) is determined. If the text associated with thevoice response does not match the data associated with the displayednumber, a determination is made that the response was incorrect, and ascore of one (1) is determined. The score may be stored, for example inmemory 105 shown in FIG. 1 , for later calculation and tabulation.

The method 600 may be repeated while the patient is holding thecomputing device 100 in their left hand. The displayed and spokeninstruction may be adjusted to reflect that the left hand should beused. The displayed number may be changed, for example, the number 2 maybe displayed.

FIG. 7 is a diagram of an example display 700 generated by the numberdetection module 230 shown in FIG. 2 . As shown in FIG. 7 , a number 710is displayed. The number 710, in this example, is three (3). In thisexample, the text instruction 720 is also displayed. Also shown in FIG.7 is an indication 730 that the one or more microphones 135 areaccessible and ready to obtain a voice response.

FIG. 8 is a flow diagram of another example of a method 800 for strokeself-detection in accordance with embodiments of this disclosure. Themethod 800 may be performed by the processor 110 shown in FIGS. 1 and 2. In this example, the method 800 may be performed by the smiledetection module 240 shown in FIG. 2 .

As shown in FIG. 8 , the method 800 includes displaying and speaking 805an instruction. For example, the smile detection module 240 may causethe processor 110 to send a signal to the display 115 shown in FIG. 1 todisplay text of the instruction, send a signal to the one or morespeakers 135 shown in FIG. 1 to speak the instruction, or both. In anexample, the text “Face forward and look directly at the camera. Smileand show your teeth for 3 seconds.” may be displayed on the display 115and the audible phrase “Face forward and look directly at the camera.Smile and show your teeth for 3 seconds.” may be emitted from the one ormore speakers 135.

The smile detection module 240 may cause the processor 110 to send asignal to the one or more image sensors 140 shown in FIG. 1 to obtainimages 810 of the face of the patient. In some implementations, theobtained images may be stored in the memory 105 shown in FIG. 1 . Theprocessor 110 may perform face detection 815 on the obtained images. Ifa face is detected, the processor 110 may perform face landmarkdetection 820 to determine if a mouth is detected. If a mouth isdetected, the processor 110 may determine 825 if a smile is detected.The processor 110 may use ML models for tasks such as classification orobject detection. The processor 110 may be configured to track multipleobjects, such as a patient's mouth or eyes, or rectangles in theobtained images.

If a smile is detected, the processor 110 may initiate 835 a countdowntimer. The countdown timer may be displayed on the display 115 showingthe time duration remaining. The countdown timer may be spoken such thatit is emitted from the one or more speakers 135.

The smile detection module 240 may cause the processor 110 to send asignal to the one or more image sensors 140 shown in FIG. 1 to recordimages 840 to obtain a video recording of the face of the patient. Insome implementations, the video recording may be stored in the memory105 shown in FIG. 1 .

The processor 110 may obtain images from the video recording and detectand segment 845 objects of the patient's face, for example, eyes,eyebrows, mouth, nose, or any combination thereof. Detecting andsegmenting objects of the patient's face may include detecting alldetectable two-dimensional (2D) face landmarks and regions, and exposingthe face landmarks and regions as properties. The coordinates of theface landmarks may be normalized to the dimensions of a face boundingbox, with the origin at the bounding box's lower-left corner. An imagepoint function may be used to convert normalized face landmark pointsinto absolute points within the coordinate system of the image or frame.The processor 110 may perform face detection, face landmark detection,and segmentation using any open source technique.

Once the mouth is segmented and the timer has expired 850, the processor110 may determine 855 a pixel distortion distance. In an example, theprocessor 110 may detect a corner of the mouth in one or more frames ofthe video recording. The video recording may be for any duration. Insome examples, the video recording duration may be for 1-3 seconds ormore. The corner of the mouth may be determined based on a lip edge. Foreach frame that the corner of the mouth is detected, the processor 110calculates 440 an absolute point within the coordinate system of theframe for that corner of the mouth. The processor 110 then compares theabsolute points between two frames. The difference between these twopoints is the pixel distortion distance.

The smile detection module 240 may cause the processor 110 to determine860 a score, for example, a stroke self-detection score, based on thecalculated pixel distortion distance. The score may be referred to as asmile score or a smile detection score. In an example, the processor 110may determine a score of zero (0) if the pixel distance is less than 2pixels. If the pixel distance is greater than 2 pixels and less than 5pixels, the processor may determine a score of one (1). If the pixeldistance is greater than 5 pixels and less than 9 pixels, the processor110 may determine a score of two (2). If the pixel distance is greaterthan 9 pixels, the processor 110 may determine a score of three (3). Thescore may be stored, for example in memory 105 shown in FIG. 1 , forlater calculation and tabulation.

In another example, the processor 110 may determine a total score forsmile distortion by determining the position of the corner of the lipsin relation to each other. A distortion of less than 0.02 is considerednormal and thus the patient receives a score of zero (0). A distortionbetween 0.02 and 0.05 results in the patient receiving a score of 1. Adistortion between 0.05 and 0.09 results in the patient receiving ascore of two (2). A distortion greater than 0.09 results in the patientreceiving a score of three (3). Similar to the eye movement detection,the software running on the processor 110 detects the patient's mouthand places a bounding box around the mouth. The y data points arerecorded, which track where the outermost edges of the lips are inrelation to the bounding box surrounding the mouth. The origin (0.0,0.0) is at the lower-left corner of the bounding box, and theupper-right corner of the box is (1.0, 1.0), similar to an (x, y)coordinate system.

FIG. 9 is a diagram of an example of an image 900 used for smiledetection. Detecting a smile may include detecting all detectabletwo-dimensional (2D) face landmarks and regions, and exposing the facelandmarks and regions as properties. As shown in FIG. 9 , thecoordinates of the face landmarks may be normalized to the dimensions ofa face bounding box 910, with the origin at the bounding box'slower-left corner. An image point function may be used to convertnormalized face landmark points into absolute points within thecoordinate system of the image or frame. As shown in FIG. 9 , detectedobjects (shown in bounding boxes as dashed lines) of the patient's facemay include, for example, eyes 920, eyebrows 930, nose 940, and mouth950.

As shown in FIG. 9 , the mouth 950 includes a left corner 960 and aright corner 970. The left corner 960, the right corner 970, or both,may be used to determine pixel distortion distance as described in FIG.8 above.

FIG. 10 is a flow diagram of another example of a method 1000 for strokeself-detection in accordance with embodiments of this disclosure. Themethod 1000 may be performed by the processor 110 shown in FIGS. 1 and 2. In this example, the method 1000 may be performed by the arm motiondetection module 250 shown in FIG. 2 , the leg motion detection module260 shown in FIG. 2 , or both.

As shown in FIG. 10 , the method 1000 includes displaying and speaking1010 an instruction. The method 1000 may be used to test the right armof the patient, the left arm of the patient, the right leg of thepatient, the left leg of the patient, or any combination thereof. Thearm motion detection module 250 or the leg motion detection module 260may cause the processor 110 to send a signal to the display 115 shown inFIG. 1 to display text of the instruction, send a signal to the one ormore speakers 135 shown in FIG. 1 to speak the instruction, or both. Inan example for testing motion of the patient's right arm, the text“Place the phone in your right hand. With your right arm outstretched,raise your right arm in the air. Hold your right arm there for 10seconds.” may be displayed on the display 115 and the audible phrase“Place the phone in your right hand. With your right arm outstretched,raise your right arm in the air. Hold your right arm there for 10seconds.” may be emitted from the one or more speakers 135. In anexample for testing motion of the patient's right leg, the text “Placeyour phone in your right hand and hold the phone on your right knee.Raise your right knee in the air. Hold it there for 5 seconds.” may bedisplayed on the display 115 and the audible phrase “Place your phone inyour right hand and hold the phone on your right knee. Raise your rightknee in the air. Hold it there for 5 seconds.” may be emitted from theone or more speakers 135.

The processor 110 may determine 1020 whether the arm or leg is raisedbased on accelerometer data, image capture data, or both. Theaccelerometer data is recorded every half second. Each time a motionrecording occurs, the total count is incremented by one (1). Theprocessor 110 tracks movement patterns based on the number of counts andthe value of the numbers. If the processor 110 determines that the armor leg is raised, the processor 110 may initiate 1030 a countdown timer.The countdown timer may be displayed on the display 115 showing the timeduration remaining. The countdown timer may be spoken such that it isemitted from the one or more speakers 135.

The arm motion detection module 250 or the leg motion detection module260 may cause the processor to obtain 1040 accelerometer data from theaccelerometer 130 shown in FIG. 1 . In some embodiments, the processormay also obtain image capture data in this step. Accelerometer data mayinclude acceleration data, vibration data, orientation data, or anycombination thereof. The processor 110 is configured to track the motionof the patient's arm or leg based on the accelerometer data, the imagecapture data, or both. For example, the processor 110 may be configuredto determine whether the patient raises their arm or leg, whether thearm or leg drifts downward, whether the arm or leg is raised but notable to remain motionless in the air, whether the arm or leg is able tomove at all, or any combination thereof, based on the accelerometerdata, the image capture data, or both. The processor 110 may performmotion tracking using any open source motion tracking technique. In someembodiments, the processor 110 may perform motion tracking using machinelearning techniques.

The processor 110 may determine 1050 whether the countdown timer hasexpired. If the countdown timer has expired, the arm motion detectionmodule 250 or the leg motion detection module 260 may cause theprocessor 110 to determine 1060 a score, for example, a strokeself-detection score, based on the accelerometer data, the image capturedata, or both. The score may be referred to as a leg motion score, a legmotion detection score, an arm motion score, an arm motion detectionscore, a limb motion score, or a limb motion detection score. In anexample, if the accelerometer data, the image capture data, or both,indicate that the arm or leg is raised and maintained in the air for thecountdown timer duration, the processor 110 may determine a score ofzero (0). If accelerometer data, the image capture data, or both,indicate that the arm or the leg drifts downwards before the expirationof the countdown timer, the processor 110 may determine a score of one(1). If the accelerometer data, the image capture data, or both,indicates that the arm or leg is raised but not motionless (i.e., thearm or the leg is shaking) in the air for the duration of the countdowntimer, the processor 110 may determine a score of two (2). If theaccelerometer data, the image capture data, or both, indicate that thearm or the leg is not raised (i.e., the user is unable to lift the armor the leg), the processor 110 may determine a score of three (3). Ifthe accelerometer data, the image capture data, or both, indicate thatthe arm or the leg is motionless (i.e., the user is unable to move thearm or the leg at all), the processor 110 may determine a score of four(4). The method 1000 may be performed for each arm and each leg of thepatient, and each limb of the patient may be scored accordingly. Thescores may be stored, for example in memory 105 shown in FIG. 1 , forlater calculation and tabulation.

In another example, for arms, if measurements have less than 22 counts,the patient likely failed the test, i.e., the processor did not recordfor the full time, meaning that the patient did not complete the test.At shoulder height, the accelerometer data reads 0.0. When theaccelerometer data reads between −21.5 and 21.5, the patient has theirarm held at an acceptable height for conducting the test. If theaccelerometer data reads less than −21.5, the patient is holding theirarm too high. If the accelerometer data reads greater than 21.5, thepatient is holding their arm too low.

For arms, if 16 or more of the measurements are within the acceptablerange, the patient receives a score of zero (0). If 6 to 15 of themeasurements are within the acceptable range, the patient receives ascore of one (1). If 1 to 5 of the measurements are within theacceptable range, the patient receives a score of two (2). If nomeasurements are within the acceptable range, the patient receives ascore of three (3).

For legs, if the measurements have 12 counts, the patient hassuccessfully completed the test on the first attempt. When the patient'sknee is 6 inches above the ground, the adjusted accelerometer data reads0.0. When the accelerometer data reads between −16.5 and 13.5, thepatient has their leg at an acceptable height for conducting the test.If the accelerometer data reads less than −16.5, the patient has theirknee lifted too high above the ground. If the accelerometer data readsgreater than 13.5, the patient has their knee lifted too low.

For legs, if 7 or more measurements are within the acceptable range, thepatient receives a score of zero (0). If 3 to 6 of the measurements arewithin the acceptable range, the patient receives a score of one (1). If1 to 2 measurements are within the acceptable range, the patientreceives a score of two (2). If no measurements are within theacceptable range, the patient receives a score of three (3).

FIG. 11 is a flow diagram of another example of a method 1100 for strokeself-detection in accordance with embodiments of this disclosure. Themethod 1100 may be performed by the processor 110 shown in FIGS. 1 and 2. In this example, the method 1100 may be performed by the vibrationdetection module 270 shown in FIG. 2 .

As shown in FIG. 11 , the vibration detection module 270 may beconfigured to cause the processor 110 to send a signal to the one ormore motors 120 shown in FIG. 1 to pulse 1110 the one or more motors120. Pulsing the one or more motors 120 will cause the computing device100 to vibrate. The one or more motors 120 may be pulsed to vibrate thecomputing device 100 in any vibration pattern, for any duration of time,and for any number of cycles. In an example, the one or more motors 120may be pulsed such that the computing device 100 vibrates for onesecond, pauses for one second, and then repeats this vibration patterntwo times (i.e., for two additional cycles).

The method 1100 includes displaying and speaking 1120 an instruction.The vibration detection module 270 may cause the processor 110 to send asignal to the display 115 shown in FIG. 1 to display text of theinstruction, send a signal to the one or more speakers 135 shown in FIG.1 to speak the instruction, or both. In an example, the text “Can youfeel this vibration?” may be displayed on the display 115 and theaudible phrase “Can you feel this vibration?” may be emitted from theone or more speakers 135.

The method 1100 includes receiving 1130 a voice response. The voiceresponse may be received by the one or more microphones 145 shown inFIG. 1 . The voice response may be processed using any open sourcespeech recognition technique. The voice response may be received as anaudible phrase such as, for example, “yes,” “affirmative,” “no,” or“negative.” The one or more microphones 145 may be configured totransmit a signal associated with the voice response to the processor110. The vibration detection module 230 may cause the processor 110 toconvert the received signal associated with the voice response to textand store the text data associated with the voice response.

The method 1100 includes determining 1140 a score, for example, a strokeself-detection score. The score may be referred to as a vibration scoreor a vibration detection score. The score may be based on adetermination of whether the text associated with the voice responsematches the data associated with a positive response or a negativeresponse. If the text associated with the voice response matches thedata associated with a positive response, a determination is made thatthe response was correct, and a score of zero (0) is determined. If thetext associated with the voice response matches the data associated witha negative response, a determination is made that the response wasincorrect, and a score of one (1) is determined. The score may bestored, for example in memory 105 shown in FIG. 1 , for latercalculation and tabulation.

FIG. 12 is a flow diagram of another example of a method 1200 for strokeself-detection in accordance with embodiments of this disclosure. Themethod 1200 may be performed by the processor 110 shown in FIGS. 1 and 2. In this example, the method 1200 may be performed by the objectdetection module 280 shown in FIG. 2 .

The object detection module 280 may cause the processor 110 to send asignal to the display 115 shown in FIG. 1 to display 1210 one or moreobjects. The one or more objects may be based on the National Instituteof Health (NIH) stroke scale. For example, the objects may be images ofa glove, a key, a cactus, a chair, a hammock, and a feather.

The method 1200 includes displaying and speaking 1220 an instruction.The object detection module 280 may cause the processor 110 to send asignal to the display 115 shown in FIG. 1 to display text of theinstruction, send a signal to the one or more speakers 135 shown in FIG.1 to speak the instruction, or both. The objects may be displayedindividually, in one or more groups, or all together. The objects may bedisplayed with or without the text instruction. In an example, the text“Name these objects.” may be displayed along with the objects on thedisplay 115 and the audible phrase “Name these objects.” may be emittedfrom the one or more speakers 135.

The method 1200 includes receiving 1230 a voice response. The voiceresponse may be received by the one or more microphones 145 shown inFIG. 1 . The voice response may be processed using any open sourcespeech recognition technique. The voice response may be received as anaudible phrase such as, for example, “glove,” “key,” “hand,” or“cactus.” The one or more microphones 145 may be configured to transmita signal associated with the voice response to the processor 110. Theobject detection module 280 may cause the processor 110 to convert thereceived signal associated with the voice response to text and comparethe text data associated with the voice response to data associated witheach displayed object.

The method 1200 includes determining 1240 a score, for example, a strokeself-detection score. The score may be referred to as an object score oran object detection score. The score may be based on a determination ofwhether the text associated with the voice response matches the dataassociated with a respective displayed object. If the text associatedwith the voice response matches the data associated with a respectivedisplayed object, a determination is made that the response was correct.If the text associated with the voice response does not match the dataassociated with a respective displayed object, a determination is madethat the response was incorrect. If the voice responses associated forall six displayed objects are determined to be correct, the processor110 may determine a score of zero (0). If the voice responses for 3-5displayed objects are determined to be correct, the processor 110 maydetermine a score of one (1). If the voice responses for 1-2 displayedobjects are determined to be correct, the processor 110 may determine ascore of two (2). If the voice responses for all six displayed objectsare determined to be incorrect, the processor 110 may determine a scoreof three (3). The processor 110 may be configured to recognize commonmistakes for objects and score them as incorrect, for example if thevoice response is “hand” instead of “glove,” the voice response will bemarked as an incorrect response. The score may be stored, for example inmemory 105 shown in FIG. 1 , for later calculation and tabulation.

FIG. 13 is a diagram of an example display 1300 generated by the objectdetection module 280 shown in FIG. 2 . As shown in FIG. 13 , the display1300 includes images of several objects including a key 1310, a glove1320, a chair 1330, a cactus 1340, a feather 1350, and a hammock 1360.In this example, the text instruction 1370 is also displayed. Also shownin FIG. 13 is an indication 1380 that the one or more microphones 135are accessible and ready to obtain a voice response. In this example,the display 1300 may also include an indication 1390 for a user touch orgesture input, such as a button or slider to advance to the next screen.

FIG. 14 is a flow diagram of another example of a method 1400 for strokeself-detection in accordance with embodiments of this disclosure. Themethod 1400 may be performed by the processor 110 shown in FIGS. 1 and 2. In this example, the method 1400 may be performed by the sentencedetection module 290 shown in FIG. 2 .

The sentence detection module 290 may cause the processor 110 to send asignal to the display 115 shown in FIG. 1 to display 1410 one or moresentences or phrases as text. The one or more sentences or phrases maybe based on the National Institute of Health (NIH) stroke scale. Forexample, the sentences or phrases may include “You know how,” “Down toearth,” “I got home from work,” “Near the table in the dining room,” and“They heard him speak on the radio last night.”

The method 1400 includes displaying and speaking 1420 an instruction.The sentence detection module 280 may cause the processor 110 to send asignal to the display 115 shown in FIG. 1 to display text of theinstruction, send a signal to the one or more speakers 135 shown in FIG.1 to speak the instruction, or both. The sentences or phrases may bedisplayed individually, in one or more groups, or all together. Thesentences or phrases may be displayed with or without the textinstruction. In an example, the text “Read the following sentences.” maybe displayed along with the sentences or phrases on the display 115 andthe audible phrase “Read the following sentences.” may be emitted fromthe one or more speakers 135.

The method 1400 includes receiving 1430 a voice response. The voiceresponse may be received by the one or more microphones 145 shown inFIG. 1 . The voice response may be processed using any open sourcespeech recognition technique. The voice response may be received as anaudible phrase such as, for example, “Near the table in the dining room”or “Down to earth.” The one or more microphones 145 may be configured totransmit a signal associated with the voice response to the processor110. The sentence detection module 290 may cause the processor 110 toconvert the received signal associated with the voice response to textand compare the text data associated with the voice response to dataassociated with each sentence or phrase.

The method 1400 includes determining 1440 a score, for example, a strokeself-detection score. The score may be referred to as a sentence scoreor a sentence detection score. The score may be based on a determinationof whether the text associated with the voice response matches the dataassociated with a respective displayed sentence or phrase. The processor110 is configured to detect whether any words in the voice responses areunclear, slurred, or disorganized (e.g., incorrect words, words inincorrect order, gaps of greater than three seconds between words). Ifthe text associated with the voice response matches the data associatedwith a respective displayed sentence or phrase, a determination is madethat the response was correct. If the text associated with the voiceresponse does not match the data associated with a respective displayedsentence or phrase (i.e., one or more words does not match), adetermination is made that the response was incorrect. If the voiceresponses associated with all the displayed sentences or phrases aredetermined to be correct, the processor 110 may determine a score ofzero (0). If any words in the voice responses are unclear, slurred, ordisorganized, the processor 110 may determine a score of one (1). If thevoice responses for all the displayed sentences or phrases aredetermined to be incorrect, the processor 110 may determine a score oftwo (2). The score may be stored, for example in memory 105 shown inFIG. 1 , for later calculation and tabulation.

FIG. 15 is a flow diagram of an example scoring method 1500 that may beused by any of the embodiments disclosed herein. As shown in FIG. 15 ,the method 1500 includes determining whether the current use of themethod is an initial use. The initial use may be more than one use, forexample, the initial use may include the first 3 to 10 uses of themethod. If it is determined that the current use is the initial use orfits the criteria of an initial use, the determined score is stored 1520as a baseline. The baseline may be an average of two or more determinedscores. If it is determined that the current use is not the initial useor does not fit the criteria of an initial use, the method 1500 includescomparing the determined score to the baseline.

The method 1500 includes determining 1540 whether the difference inscore relative to the baseline is greater than a threshold. If thedifference in score relative to the baseline is above a threshold, themethod includes generating and transmitting 1550 an alert that indicatesthat the patient may be experiencing a stroke. The alert may betransmitted to the patient's primary care physician, emergency contact,emergency medical services, telemedicine provider, or any combinationthereof. In some embodiments, the method 1500 may include requestingpermission from the patient to transmit the alert. If the difference inscore relative to the baseline is below the threshold, the methodincludes storing 1560 the score. In some embodiments, the scores fromone or more methods may be tabulated and displayed numerically andqualitatively on the display 115.

One or more methods described herein may be combined, and the scores maybe summed to determine an overall score. The overall score may be usedto determine whether to contact the patient's primary care physician,emergency contact, emergency medical services, telemedicine provider, orany combination thereof. The scores may be tabulated and displayednumerically and qualitatively on the display.

Although some embodiments herein refer to methods, it will beappreciated by one skilled in the art that they may also be embodied asa system or computer program product. Accordingly, aspects of thepresent invention may take the form of an entirely hardware embodiment,an entirely software embodiment (including firmware, resident software,micro-code, etc.) or an embodiment combining software and hardwareaspects that may all generally be referred to herein as a “processor,”“device,” or “system.” Furthermore, aspects of the present invention maytake the form of a computer program product embodied in one or morecomputer readable mediums having computer readable program code embodiedthereon. Any combination of one or more computer readable mediums may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium include the following: an electrical connection havingone or more wires, a portable computer diskette, a hard disk, a randomaccess memory (RAM), a read-only memory (ROM), an erasable programmableread-only memory (EPROM or Flash memory), an optical fiber, a portablecompact disc read-only memory (CD-ROM), an optical storage device, amagnetic storage device, or any suitable combination of the foregoing.In the context of this document, a computer readable storage medium maybe any tangible medium that can contain, or store a program for use byor in connection with an instruction execution system, apparatus, ordevice.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to CDs, DVDs,wireless, wireline, optical fiber cable, RF, etc., or any suitablecombination of the foregoing.

Computer program code for carrying out operations for aspects of thepresent invention may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java, Smalltalk, C++ or the like and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Aspects of the present invention are described below with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks. These computerprogram instructions may also be stored in a computer readable mediumthat can direct a computer, other programmable data processingapparatus, or other devices to function in a particular manner, suchthat the instructions stored in the computer readable medium produce anarticle of manufacture including instructions which implement thefunction/act specified in the flowchart and/or block diagram block orblocks. The computer program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other devicesto cause a series of operational steps to be performed on the computer,other programmable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. The flowcharts and block diagrams in thefigures illustrate the architecture, functionality, and operation ofpossible implementations of systems, methods and computer programproducts according to various embodiments of the present invention. Inthis regard, each block in the flowchart or block diagrams may representa module, segment, or portion of code, which comprises one or moreexecutable instructions for implementing the specified logicalfunction(s). It should also be noted that, in some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts, or combinations of special purpose hardware andcomputer instructions.

While the disclosure has been described in connection with certainembodiments, it is to be understood that the disclosure is not to belimited to the disclosed embodiments but, on the contrary, is intendedto cover various modifications and equivalent arrangements includedwithin the scope of the appended claims, which scope is to be accordedthe broadest interpretation so as to encompass all such modificationsand equivalent structures as is permitted under the law.

What is claimed is:
 1. A computing device for stroke self-detection, the computing device comprising: a display configured to display an instruction; a plurality of sensors that include an accelerometer, a camera, and a microphone; a memory; and a processor configured to: obtain sensor data based on the instruction, wherein the sensor data includes accelerometer data from the accelerometer, image capture data from the camera, and microphone data from the microphone that includes voice data associated with a patient response; convert the voice data to text data; determine a stroke self-detection score based on the obtained sensor data, wherein the determination of the stroke self-detection score includes a comparison of the text data to data associated with the instruction and a determination of a cognitive score, wherein the instruction includes a request for patient age, and wherein the determination of the cognitive score comprises: a determination that the cognitive score is zero (0) on a condition that the text data matches the data associated with the instruction; and a determination that the cognitive score is one (1) on a condition that the text data does not match the data associated with the instruction; store the stroke self-detection score in the memory; display the stroke self-detection score on the display as a result summary; and on a condition that the stroke self-detection score is above a threshold, transmit an alert.
 2. The computing device of claim 1, further comprising a speaker, wherein the processor is further configured to: convert the instruction to speech data; and emit the speech data as a voice instruction via the speaker.
 3. The computing device of claim 1, wherein the determination of the stroke self-detection score includes a determination of a cognitive score, wherein the instruction includes a request for a current month, and wherein the determination of the cognitive score comprises: a determination that the cognitive score is zero (0) on a condition that the text data matches the data associated with the instruction; and a determination that the cognitive score is one (1) on a condition that the text data does not match the data associated with the instruction.
 4. The computing device of claim 1, wherein the determination of the stroke self-detection score includes a determination of a number detection score, wherein the instruction includes an image of an object, and wherein the determination of the number detection score comprises: a determination that the number detection score is zero (0) on a condition that the text data matches the data associated with the instruction; and a determination that the number detection score is one (1) on a condition that the text data does not match the data associated with the instruction.
 5. The computing device of claim 1, further comprising a motor configured to pulse, wherein the determination of the stroke self-detection score includes a determination of a vibration score, wherein the instruction includes a request for a response associated with a vibration caused by the motor, and wherein the determination of the vibration score comprises: a determination that the vibration score is zero (0) on a condition that the text data matches data associated with a positive response; and a determination that the vibration score is one (1) on a condition that the text data matches data associated with a negative response.
 6. The computing device of claim 1, wherein the determination of the stroke self-detection score includes a determination of an object detection score, wherein the instruction includes an image of a plurality of objects, and wherein the determination of the object detection score comprises: a determination that the object detection score is zero (0) on a condition that the text data matches data associated with each of the plurality of objects; a determination that the object detection score is two (2) on a condition that the text data matches data associated with at least one of the plurality of objects; a determination that the object detection score is one (1) on a condition that the text data matches data associated with at least three of the plurality of objects; and a determination that the object detection score is three (3) on a condition that the text data does not match data associated with any of the plurality of objects.
 7. The computing device of claim 1, wherein the determination of the stroke self-detection score includes a determination of a sentence detection score, wherein the instruction includes a plurality of sentences, and wherein the determination of the sentence detection score comprises: a determination that the sentence detection score is zero (0) on a condition that the text data matches data associated with each of the plurality of sentences; a determination that the sentence detection score is one (1) on a condition that the text data does not match data associated with at least one of the plurality of sentences; and a determination that the sentence detection score is two (2) on a condition that the text data does not match data associated with any of the plurality of sentences.
 8. The computing device of claim 1, wherein the image capture data includes a frame that includes a face, the processor further configured to: perform face detection to identify an eye on the face; detect a pupil position of the eye on the frame; calculate a relative pupil position within a bounding box surrounding the eye.
 9. The computing device of claim 8, wherein the determination of the stroke self-detection score includes a determination of an eye movement score based on a right eye passing score of 0.1 and a left eye passing score of 0.07, wherein the determination of the eye movement score comprises: a determination that the eye movement score is zero (0) on a condition that the right eye passing score and the left eye passing score are met; a determination that the eye movement score is one (1) on a condition that a left eye distance is greater than the left eye passing score or a right eye distance is greater than the right eye passing score; a determination that the eye movement score is two (2) on a condition that the left eye distance is greater than the left eye passing score and the right eye distance is greater than the right eye passing score; and a determination that the eye movement score is two (2) on a condition that one eye receives a passing score and the other eye receives a failing score.
 10. The computing device of claim 1, wherein the image capture data includes a first frame that includes a face and a second frame that includes the face, the processor further configured to: perform face detection to identify a mouth on the face, wherein the mouth has a left corner and a right corner; detect a smile based on the right corner or the left corner; calculate a first absolute point on the first frame, wherein the first absolute point is based on the right corner or the left corner; calculate a second absolute point on the second frame, wherein the second absolute point is based on a same corner as the first absolute point; and calculate a pixel distortion value, wherein the pixel distortion value is based on a difference of a measurement between the first absolute point and the second absolute point.
 11. The computing device of claim 10, wherein the determination of the stroke self-detection score includes a determination of a smile detection score, wherein the determination of the smile detection score comprises: a determination that the smile detection score is zero (0) on a condition that the pixel distortion value is less than 0.02; a determination that the smile detection score is one (1) on a condition that the pixel distortion value is between 0.02 and 0.05; a determination that the smile detection score is two (2) on a condition that the pixel distortion value is between 0.05 and 0.09; and a determination that the smile detection score is three (3) on a condition that the pixel distortion value is greater than 0.09.
 12. The computing device of claim 1, wherein the determination of the stroke self-detection score includes a determination of an arm motion detection score, wherein the determination of the arm motion detection score comprises: a determination that the arm motion detection score is zero (0) on a condition that the accelerometer data indicates that an arm is maintained in an elevated position for a predetermined duration; a determination that the arm motion detection score is one (1) on a condition that the accelerometer data indicates that the arm drifted downward prior to an expiration of the predetermined duration; a determination that the arm motion detection score is two (2) on a condition that the accelerometer data indicates that the arm is elevated and shaking; a determination that the arm motion detection score is three (3) on a condition that the accelerometer data indicates that the arm is not elevated; and a determination that the arm motion detection score is four (4) on a condition that the accelerometer data indicates that the arm is motionless.
 13. The computing device of claim 1, wherein the determination of the stroke self-detection score includes a determination of a leg motion detection score, wherein the determination of the leg motion detection score comprises: a determination that the leg motion detection score is zero (0) on a condition that the accelerometer data indicates that a leg is maintained in an elevated position for a predetermined duration; a determination that the leg motion detection score is one (1) on a condition that the accelerometer data indicates that the leg drifted downward prior to an expiration of the predetermined duration; a determination that the leg motion detection score is two (2) on a condition that the accelerometer data indicates that the leg is elevated and shaking; a determination that the leg motion detection score is three (3) on a condition that the accelerometer data indicates that the leg is not elevated; and a determination that the leg motion detection score is four (4) on a condition that the accelerometer data indicates that the leg is motionless.
 14. A computing device for stroke self-detection, the computing device comprising: a display configured to display an instruction; a microphone; a memory; and a processor configured to: obtain microphone data from the microphone, wherein the microphone data includes voice data associated with a patient response; convert the voice data to text data; determine a stroke self-detection score based on a comparison of the text data to data associated with the instruction and a determination of a cognitive score, wherein the instruction includes a request for a current month, and wherein the determination of the cognitive score comprises: a determination that the cognitive score is zero (0) on a condition that the text data matches the data associated with the instruction; and a determination that the cognitive score is one (1) on a condition that the text data does not match the data associated with the instruction; store the stroke self-detection score in the memory; display the stroke self-detection score on the display as a result summary; and on a condition that the stroke self-detection score is above a threshold, transmit an alert.
 15. The computing device of claim 14, wherein the determination of the stroke self-detection score includes a determination of a number detection score, wherein the instruction includes an image of a number, and wherein the determination of the number detection score comprises: a determination that the number detection score is zero (0) on a condition that the text data matches the data associated with the instruction; and a determination that the number detection score is one (1) on a condition that the text data does not match the data associated with the instruction.
 16. The computing device of claim 14, further comprising a motor, wherein the determination of the stroke self-detection score includes a determination of a vibration score, wherein the instruction includes a request for a response associated with a vibration caused by the motor, and wherein the determination of the vibration score comprises: a determination that the vibration score is zero (0) on a condition that the text data matches data associated with a positive response; and a determination that the vibration score is one (1) on a condition that the text data matches data associated with a negative response.
 17. The computing device of claim 14, wherein the determination of the stroke self-detection score includes a determination of an object detection score, wherein the instruction includes an image of a plurality of objects, and wherein the determination of the object detection score comprises: a determination that the object detection score is zero (0) on a condition that the text data matches data associated with each of the plurality of objects; a determination that the object detection score is two (2) on a condition that the text data matches data associated with at least one of the plurality of objects; a determination that the object detection score is one (1) on a condition that the text data matches data associated with at least three of the plurality of objects; and a determination that the object detection score is three (3) on a condition that the text data does not match data associated with any of the plurality of objects. 